﻿ My Geography Manual is Now Linked with the Web and the Real World Dan Cristea “Alexandru Ioan Cuza” University of Iași, Faculty of Computer Science Romanian Academy, Instute for Computer Science dcristea@info uaic ro Acknowledgements… MappingBooks – Enter the book! Evade from the book in the virtual and real world! a PN-II project (July 2014 – September 2017) ﬁnanced by the Romanian Ministry of Educaon and Research 2 IDAACS-2017, University Politehnica of Bucharest, 22 Sept I like traveling and reading… 3 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Going out of the book… Çelebi Mh , Maç Sk, Beyoğlu, Turkey to Çukur Cuma Cd, Beyoğlu, Turkey - Google Maps10/3/1310/3/13 8:13 PMKatip Çelebi Mh , Maç Sk, Beyoğlu, Turkey to Çukur Cuma Cd, Beyoğlu, Turkey - Google Maps 8:13 PMKatip Directions to Çukur Cuma Cd, Beyo!lu, Turkey 400 m – about 4 mins Walking directions are in beta Use caution – This route may be missing sidewalks or pedestrian paths Katip Çelebi Mh , Maç Sk, Beyo!lu, Turkey" Çukur Cuma Cd, Beyo!lu, Turkey" 1 Head southwest on Maç Sk toward Baltacı Çkgo 75 m About 47 secstotal 75 m These directions are for planning purposes only You may find that construction projects, traffic, weather, or other events may cause conditions to differ from the map results, and you should plan your route accordingly You must obey all signs or notices regarding your route Map data ©2013 Basarsoft 2 Turn right onto Turnacıba"ı Cdgo 28 m total 100 m 3 Turn left onto A!a Külhanı Sk (Altıpatlar Sk )go 130 m About 2 minstotal 240 m 4 Continue onto Çukur Cuma Cdgo 150 m About 1 mintotal 400 m Page 2 of 2https://maps google com/maps?f=d&source=s d&saddr=Maç+Sokak,+I…,288 55,2 369,37 281,0&layer=c&ei=OqVNUp3mE8nTtAaWr4CgCQ&pw=2 Page 1 of 2https://maps google com/maps?f=d&source=s d&saddr=Maç+Sokak,+I…,288 55,2 369,37 281,0&layer=c&ei=OqVNUp3mE8nTtAaWr4CgCQ&pw=2 4 IDAACS-2017, University Politehnica of Bucharest, 22 Sept I need help to remember all kinship relaons between characters 5 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Characters in Forsyte Saga • The old Forsytes Ann, the eldest of the family Old Jolyon, the patriarch of the family, having made a fortune in tea James, a solicitor, married to Emily, a most tranquil woman Swithin, James's twin brother with aristocrac pretensions; a bachelor Roger, "the original Forsyte" Julia (Juley), a ﬂuery dowager; Mrs Sepmus Small Hester, an old maid Nicholas, the wealthiest in the family Timothy, the most cauous man in England Susan, the married sister • The young Forsytes Young Jolyon, Old Jolyon's arsc and free-thinking son, married three mes Soames, James and Emily's son, an intense, unimaginave and possessive solicitor, married to the unhappy Irene, who later marries Young Jolyon Winifred, Soames's sister, one of the three daughters of James and Emily, married to the foppish and lethargic Montague Dare George, Roger's son, a dyed-in-the-wool mocker Francie, George's sister and Roger's daughter, emancipated from God • Their children June, Young Jolyon's deﬁant daughter from his ﬁrst marriage; engaged to an architect, Philip Bosinney, who becomes Irene's lover Jolly, Young Jolyon's son from his second marriage; dies of enteric fever during the Boer Wars Holly, Young Jolyon's daughter from his second marriage, to June's governess Jon, Young Jolyon's son from his third marriage, to Irene, Soames's ﬁrst wife Fleur, Soames's daughter from his second marriage, to a French Soho shopgirl Annee; Jon's lover; later marries a baronet, Michael Mont Val, Winifred and Montague's son; ﬁghts in the Boer Wars; marries his cousin Holly Imogen, Winifred and Montague's daughter • Others Parﬁ, Old Jolyon's butler Smither, Aunts Ann, Juley and Hester's housekeeper Warmson, James and Emily's butler Bilson, Soames's housemaid Prosper Profond, Winifred's admirer and Annee's lover 6 IDAACS-2017, University Politehnica of Bucharest, 22 Sept 7 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Bring back the book in the hands of children! • What do youngsters keep in their hands in our mes? – Tablets – Kendamas – Books? IDAACS-2017, University Politehnica of Bucharest, 22 Sept Linguiscs Linked Open Data (LLOD) a subﬁeld of Natural Language Processing - Develop techniques able to decipher the semanc content of texts - narrave lines (e g what happens and when) - semanc relaons between enes (e g genealogical trees, spaal and temporal relaons) - stascs about enes (# menons => salience, etc ) - summaries (general, focused on characters) 9 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Linguiscs Linked Open Data (LLOD) - Generaon of ontologies from collecons of scienﬁc works - applicaons that “read” science books and formalize concepts and their instances - Intelligent documentary search - Personalized assistants of a research acvity 10 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Enty linking • Challenges in enty linking: – name variaons – ambiguies – first menons – reference chains 11 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Linking enes internally 12 IDAACS-2017, University Politehnica of Bucharest, 22 Sept hps://nlptools info uaic ro The ‘QuoVadis’ corpus 13 IDAACS-2017, University Politehnica of Bucharest, 22 Sept A corpus semanc enes and relaons • Type of enes: – persons – gods – groups of persons and gods – body parts • Semanc relaons among enes of these types 14 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Relations • Anaphoric relations: co-referential; • Non-anaphoric relations: – kinship; – affective; – social 15 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Anaphoric relations • coref • coref-interpret • member-of, has-as-member (inverse) • isa, class-of (inverse) • part-of, has-as-part (inverse) • subgroup-of, has-as-subgroup (inverse) • has-name, name-of (inverse) Example: [Lygia] was unable to answer, for weeping seized [her] anew Acte gathered 12 [the maiden] to her bosom, and strove to calm [her] excitement 34 coref ; coref-interpret ; coref 16 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Kinship relations • parent-of • child-of (inverse of parent-of) • grandparent-of and grandchild-of (inverse) • sibling (symmetrical) • ant-uncle-of, nephew-of (inverse relation) • cousin-of (symmetrical) • spouse-of (symmetrical) • unknown Example: "Pardon me, Lygia For me thou art [ [of a king]] and [ [of Plautius]] “ 43 child-of ; child-of 17 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Social relations • superior-of • inferior-of • in cooperation-with • colleague-of • in competition-with • opposite-to Example: [Petronius]…but to [his] misfortune [he] 123 [Cæsar himself], hence [he] roused [his] jealousy 456 in competition-with ; coref ; coref ; coref 18 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Affective relations • love • loved-by • hate • hated by • upset • friendship • worship Example: Vinicius entered Lygia's dungeon and remained there till daylight…Both changed by degrees into sad souls with [each] [other] 12 rec-love 19 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Relaons • Anaphoric: coref John met Maria on the ski slope He raced her anafor antecedent 20 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Relaons • Anaphoric: coref John met Maria on the ski slope He raced her anafor antecedent 21 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Arguments and triggers in relaons • Kinship: parent-of … her father … source desnaon trigger 22 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Arguments and triggers in relaons • Social: inferior-of Cesar’ s principal courers … trigger desnaon source 23 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Arguments and triggers in relaons • Aﬀecve: worship Lygia dropped on her knees to implore someone else trigger desnaon source 24 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Enes Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 25 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Anaphoric relaons: coref Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 26 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Anaphoric relaons: coref Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 27 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Anaphoric relaons: coref Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 28 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Anaphoric relaons: class-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 29 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Kinship relaons: sibling Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 30 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Kinship relaons: child-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 31 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Kinship relaons: parent-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 32 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Kinship relaons: spouse-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 33 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Social relaons: inferior-of Petroniu… Vinicius was the son of his oldest sister , who years before had married his father , a man of consular dignity from the me of Tiberius 34 IDAACS-2017, University Politehnica of Bucharest, 22 Sept General stascs over the corpus • 7,281 sentences • 146,822 tokens, punctuaon included • 171,029 tokens summed up under all relaons • 24,636 enty menons • 22,301 referenal relaons • 755 AKS relaons (Aﬀecve + Kinship + Social) • 752 triggers 35 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Example: aﬀecve relaons love and worship 36 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Example: aﬀecve relaons fear-of and hate 37 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Vinicius’ links with other characters 38 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Semanc relaons involving Vinicius 39 IDAACS-2017, University Politehnica of Bucharest, 22 Sept căsătorise Marcus cu Vinicius TYPE="parent-of"> era tatăl TYPE="child-of"> acestuia ﬁul TYPE="sibling-of"> surorii TYPE="inferior-of"> sale o consul mai pe mari vremea lui ta o Tiberiu n , n TYPE="spouse-of"> A care REFERENTIAL> , cu REFERENTIAL> ani în urmă , of" /REFERENTIAL> se 40 Linking enes externally 41 IDAACS-2017, University Politehnica of Bucharest, 22 Sept MappingBooks – a bird’s view 42 MappingBooks • A MappedBook is a book connected with locaons/events in the virtual and real world and sensive to the instantaneous locaon of the reader (as seized by the telephone/tablet) • The informaon made available could possibly be diﬀerent depending on the moment and the place of the reader 43 IDAACS-2017, University Politehnica of Bucharest, 22 Sept MappingBooks • Mul-dimensional mash-ups combining textual and geographical data • Spot book menons of enes (persons and locaons) and link them in the virtual world • Make heavy use of enty linking techniques • Easy to handle interface for young readers 44 IDAACS-2017, University Politehnica of Bucharest, 22 Sept The applicaon 1) Connects menons of enes (nominal groups) => one enty = a chain of coreferenal menons 2) The knowledge base does not include any apriory records about enes => starts from scratch 3) Idenﬁes geographical relaons (distances, posions, proximies, intersecons, etc ) 4) Texts, for the me being: geography manuals 45 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Enty types in MB • Type PERSON • Type LOCATION • Type ORGANISATION • Type URL • Type TIMEX IDAACS-2017, University Politehnica of Bucharest, 22 Sept Textual realisaon of enes • Syntacc realisaon: NPs (proper nouns, common nouns, adjecves, complement PPs; but NO relave clauses) • Characterised by disncve heads – [the house on the [mountain]] • If intersected è imbricated – [the museum [Grigore Anpa]] IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • The capacity to see a text diﬀerent than a string of leers – sentence spling – tokenisaon – POS-tagging – lemmasaon – NP chunking – anaphora resoluon TEXT ANALYTICS IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • Know who’s who – recognise names and types – disambiguate names – recognise an enty in the text even if menoned by a common noun or a pronoun – use an ontology of types NAME ENTITY RECOGNITION IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • What virtual world enes are menoned in the book? – link textual menons of enes in the virtual world – decide what virtual info would be relevant to user – employ mulple sources ENTITY CROWLING IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • Fetch, process and make use of geo-data – Geographic Informaon Systems (GIS) – geographic layers GEOGRAPHY IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • Trace on a map a spaal relaon described in the book – spaal relaons detecon in text – use Google Maps-like geo-strata (actually we procured our own maps) – trace locaons and paths on maps RELATIONS DETECTION MAPS&TRAJECTORIES IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • Know where I am • What real world enes are in my proximity – detecon of my posion – computaon of distances from the menoned places – signalling “interesng” locaons in proximity DEVICE INFO IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • Mix images with generated info – locate the posion of the user (GPS) – Sense the orientaon of the camera (compass) – process images => segment, contours, recognion – decide info to be displayed AUGMENTED REALITY IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • Aracve user interfaces – analyse use cases – design dedicated user interfaces – accommodate on the screen a segment of text, a map, user’s posion, web info, etc INTERFACES IDAACS-2017, University Politehnica of Bucharest, 22 Sept Processing features • Client-server – user’s Portrait – the databases – standards and communicaon protocols CLIENT-SERVER IDAACS-2017, University Politehnica of Bucharest, 22 Sept Other issues… • RESOURCES – ﬁnd the texts – clear IPR – perform annotaon – ﬁnd other relevant linguisc data IDAACS-2017, University Politehnica of Bucharest, 22 Sept TA = Text Analytics NER = Name Entity Recognition AR = Augmented Reality EC = Entity Crowling DEV = Device Info RD = Relations Detection INT = Interfaces GEO = Geography RES = Resources M&T = Maps and Trajectories M&E = Management and Evaluation 58 IDAACS-2017, University Politehnica of Bucharest, 22 Sept What else could be added? Networking Readers • Using semanc and geographical links to form social communies of readers – if books “subscripted for” declared visible => • co-readers of B (book) – if “instantaneous locaon” also declared visible => • co-readers of B AND actual co-proximity of L (locaon) • co-readers of B AND co-track of T (trajectory) 59 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Networking Readers: enhance e-Books reading experience • Easy to imagine other ways to form communies rooted in readings – intersect common readings and aended places with levels of friendship reported by other social media, like Facebook or Twier – real-world events and enes menoned in a book associated with real-world locaons and parcular moments of the year/day – portraying the user (from accessible social media and habits of MB behavior) and matching 60 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Usage examples - I visit a city with the traveling guide in my hand - places of interest, routes, are reordered depending on my instantaneous posion 61 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Usage examples - I am a school boy, in the train going from Brașov to Sibiu… - if I open my tablet and head it towards the le side window of the train, I will see arrows showing the picks of the Făgăraș mountains, exactly as in the Geography manual 62 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Augmented reality IDAACS-2017, University Politehnica of Bucharest, 22 Sept Usage examples - I am in Paris for the 3rd me… - but only now my MB Lonely Planet guide signals me this temporary exhibion opened in the Pyramid 64 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Towards… live books • Muldimensional artefacts that combine textual, geographical, temporal, etc data • Evidence menons of persons, locaons… • Links sensible at: – the context of the menon in book – the locaon of the reader – the moment of the lecture – the personality and preferences of the reader 65 IDAACS-2017, University Politehnica of Bucharest, 22 Sept Real or virtual? Troubling quesons… • To what degree should the technology interfere with the act of literary creaon? • Virtual reality techniques are at our touch – sll, which of the descripon of percepon menoned in a book should be added to the text? • We do not want to rebuild in virtual sensaons, images, sounds… – we only want to help the reader with complementary informaon 66 IDAACS-2017, University Politehnica of Bucharest, 22 Sept MappingBooks: more acknowledgements • Undergrad students from the Faculty of Computer Science, who in the ﬁrst term of the univ year 2013-2014 have build the MB prototype as a term project in the AI course • My colleagues: Ionuț Pistol, Daniela Gîfu (Fac CS), Mihai Niculiță (Fac Geography) in UAIC • Partners in the MB project: Univ “Ștefan cel Mare” Suceava, SIVECO – Bucharest 67 IDAACS-2017, University Politehnica of Bucharest, 22 Sept References • M Colhon, D Cristea, D Gîfu (2016) Discovering Semanc Relaons within Nominals in D Trandabăț and D Gîfu (eds ): Proceedings of the Workshop on Social Media and the Web of Linked Data, RUMOUR-2015, A satellite event of EUROLAN-2015, Sibiu, Romania, July 2015, Springer Internaonal Publishing • D Cristea, D Gîfu, M Colhon, P Diac, A -D Bibiri, C Mărănduc, and L -A Scutelnicu (2015) Quo Vadis: A Corpus of Enes and Relaons In N Gala, R Rapp and G B Enguix (eds ): Language Producon, Cognion, and the Lexicon, Springer Internaonal Publishing Switzerland • D Cristea, D Gîfu, I -C Pistol, D Sﬁrnaciuc, M Niculiță (2016) A Mixed Approach in Recognising Geographical Enes in Texts in D Trandabăț and D Gîfu (eds ): Proceedings of the Workshop on Social Media and the Web of Linked Data, RUMOUR-2015, A satellite event of EUROLAN-2015, Sibiu, Romania, July 2015, Springer Internaonal Publishing • D Cristea, Ș -G Penuc (2016) A New eBook Concept and Technology Dedicated to Geographical Informaon, 18-th Internaonal Conference on Scienﬁc Research & Educaon in the Air Force – AFASES, Brașov, May • D Cristea, I -C Pistol (2014) MappingBooks: Linguisc Support For Geographical Navigaon Systems In M Colhon, A Iene, V Barbu Mitelu, D Cristea, D Tuﬁş (eds ) (2014) Proceedings of the 10th Internaonal Conference "Linguisc Resources And Tools For Processing The Romanian Language, Craiova, 18-19 September 2014", „Alexandru Ioan Cuza” University Publishing House • D Cristea, I -C Pistol, D Gîfu and D Anechitei (2016) Networking Readers: Using Semanc and Geographical Links to Enhance e-Books Reading Experience, in Proceedings of the 2nd Workshop on Social Media and the Web of Linked Data, RUMOUR 2016, together with the 8th ICCCI, September 28-30 2016, Halkidiki, Greece Thank you! 69 IDAACS-2017, University Politehnica of Bucharest, 22 Sept 